Commission V Special Interest Working Group on “Animation”

In this paper we present a method to extract 3-D information of the shape and movement of the human body using video sequences acquired with three CCD cameras. This work is part of a project aimed at developing a highly automated system to model most realistically human bodies from video sequences. Our image acquisition system is currently composed of three synchronized CCD cameras and a frame grabber which acquires a sequence of triplet images. From the video sequences, we extract two kinds of 3-D information: a three dimensional surface measurement of the visible body parts for each triplet and 3-D trajectories of points on the body. Our approach to surface measurement is based on multi-image matching, using the adaptive least squares method. A semi automated matching process determines a dense set of corresponding points in the triplets, starting from few manually selected seed points. The tracking process is also based on least squares matching techniques, thus the name LSMTA (Least Squares Matching Tracking Algorithm). The spatial correspondences between the three images of the different views and the temporal correspondences between subsequent frames are determined with a least squares matching algorithm. The advantage of this tracking process is twofold: firstly, it can track natural points, without using markers; secondly, it can also track entire surface parts on the human body. In the last case, the tracking process is applied to all the points matched in the region of interest. The result can be seen as a vector field of trajectories (position, velocity and acceleration) which can be checked with thresholds and neighborhood-based filters. The 3-D information extracted from the video sequences can be used to reconstruct the animation model of the original sequence.